Automated modification of graphical user interfaces

ABSTRACT

Information that describes one or more touch events performed on a touchscreen display to interact with a graphical user interface (GUI) of a software application is received. A touch event density map based on a frequency of touch events on the touchscreen display as a function of touch event position on the touchscreen display is generated. One or more portions of the touchscreen display that received the highest density of touch events are identified. At least one of: dimension of the one or more graphical elements of the GUI associated with the one or more identified portions of the touchscreen display, a dimension of a touchscreen display area that corresponds to the one or more graphical elements, and a position relative to the one or more graphical elements of the touchscreen display area are modified.

BACKGROUND OF THE INVENTION

The present invention relates generally to the field of touchscreencomputer systems, and more particularly, modifying graphical userinterfaces of touchscreen computer systems.

Touchscreen computer systems often implement a graphical user interface(GUI) enabling users of touchscreen computer systems to interact withone or more applications executed on the touch-display computer systems.For example, GUIs may include virtual keyboards which emulate physicalcomputer keyboards. Virtual keyboards enable users of touch-displaycomputer systems to input alphanumeric data using a text entryinterface. Users that use touch-display computer systems may havevarying physical capabilities, physical limitations, and/or personalpreferences when interacting with one or more applications executed on atouch-display computer system.

SUMMARY

An automated modification of graphical user interfaces (GUI) isprovided. Information that describes one or more touch events performedon a touchscreen display to interact with a graphical user interface(GUI) of a software application is received. A touch event density mapbased on a frequency of touch events on the touchscreen display as afunction of touch event position on the touchscreen display isgenerated. One or more portions of the touchscreen display that receiveda highest density of touch events are identified. One or more graphicalelements of the GUI associated with the one or more identified portionsof the touchscreen display are identified. Responsive to determiningthat the one or more graphical elements of the GUI associated with theone or more identified portions of the touchscreen display do not alignwith a threshold alignment value, at least one of: a dimension of theone or more graphical elements of the GUI associated with the one ormore identified portions of the touchscreen display, a dimension of atouchscreen display area that corresponds to the one or more graphicalelements, and a position relative to the one or more graphical elementsof the touchscreen display area are modified.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a graphical user interface modificationenvironment, in accordance with an embodiment of the present invention;

FIG. 2 is a flowchart illustrating operational steps for modifying agraphical user interface, in accordance with an embodiment of thepresent invention;

FIGS. 3A-3C are block diagrams of graphical elements of a graphical userinterface, in accordance with an embodiment of the present invention;

FIG. 4 is a block diagram of internal and external components of thecomputer systems of FIG. 1, in accordance with an embodiment of thepresent invention;

FIG. 5 depicts a cloud computing environment according to an embodimentof the present invention; and

FIG. 6 depicts abstraction model layers, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention provide systems, methods, andcomputer program products for automatically modifying graphical userinterfaces implemented by touch-display computer systems. Graphical userinterfaces, for example, virtual keyboards, may be automatically modifyfor each user interacting with an application executed on touch-displaycomputer systems, based on user input and analyzed touchscreen touchevents.

FIG. 1 is a block diagram of graphical user interface (GUI) modificationenvironment 100, in accordance with an embodiment of the presentinvention. GUI modification environment 100 includes client computersystem 110 and storage computer system 130. Client computer system 110and storage computer system 130 can be a desktop computers, laptopcomputers, specialized computer servers, or any other computer systemsknown in the art, in accordance with an embodiment of the invention. Incertain embodiments, client computer system 110 and storage computersystem 130 represent computer systems utilizing clustered computers andcomponents to act as a single pool of seamless resources when accessedthrough network 120. In certain embodiments, client computer system 110and storage computer system 130 represent virtual machines. In general,client computer system 110 and storage computer system 130 arerepresentative of any electronic devices, or combination of electronicdevices, capable of executing machine-readable program instructions inaccordance with an embodiment of the invention, as described in greaterdetail with regard to FIG. 4. In other embodiments, client computersystem 110 and storage computer system 130 may be implemented in a cloudcomputing environment, as described in relation to FIGS. 5 and 6.

Client computer system 110 represents a platform by which one or moreusers can interact with application 116, via GUI 114 presented ontouchscreen display 118. Client computer system 110 includes analysisengine 112, GUI 114, application 116, and touchscreen display 118. Inthis embodiment, a user of client computer system 110 may useapplication 116 by interacting with GUI 114 which is presented ontouchscreen display 118. The phrase, “touch event” as used herein,refers to a user input (e.g., touch, tapped, swipe, etc.) on touchscreendisplay 118 performed by a user of client computer system 110 tointeract with a portion of GUI 114 and is associated with touch eventinformation. For example, a user may perform a touch event ontouchscreen display 118 to interact with a specific graphical element ofGUI 114 (e.g., a Home button). Application 116 receives an indicationfrom GUI 114 that user selection of the Home button has occurred, andresponds appropriately.

Analysis engine 112 receives touch event information from, for example,GUI 114 via an API, that identifies the specific graphical element ofGUI 114 touched, and the shape of the physical area touched ontouchscreen display 118. Analysis engine 112 may generate a density map,based on the touch event information, to modify GUI 114. The phrase“density map” as used herein, refers to a representation of GUI 114 thatidentifies portions of touchscreen display 118 most frequently receivingtouch events (e.g., touched, tapped, etc.). Stated differently, adensity map is based on the frequency of touch events on touchscreendisplay 118 as a function of touch event position on touchscreen display118. A density map may be generated for each selectable graphicalelement of GUI 114. The density map may combine touch events, such thatanalysis engine 112 can use the density map to determine a manner inwhich to modify a boundary of a graphical element of GUI 114, such thatthe modified graphical element is modified based on the analyzed touchevent information. For example, GUI 114 may include four square-shapedgraphical elements, such as touch selectable buttons. In this instance,a density map can be generated by analysis engine 112 to graphicallyrepresent one or more interactions between a user and the foursquare-shaped graphical elements. The generated density map may revealthat each of the one or more touch events between the user and the foursquare-shaped graphical elements takes the form of a circle.Accordingly, analysis engine 112 may modify the boundaries of the foursquare-shaped graphical elements and the associated areas of touchscreendisplay 118, such that an area of each of the four square-shapedgraphical elements matches the circle area, as presented by the densitymap. Alternatively, the areas of touchscreen display 118 associated withthe four square-shaped graphical elements of GUI 114 may be modified tomatch the circle areas determined by the density map, while notmodifying the visual display of the four square-shaped graphicalelements of GUI 114.

GUI 114 represents an interface that includes one or more interactivegraphical elements enabling a user of client computer system 110 to useapplication 116 executed on client computer system 110. For example, theuser of client computer system 110 may perform various tasks ofapplication 116 by interacting with GUI 114. In one embodiment, GUI 114may include a virtual keyboard, which can be arranged similar to astandard QWERTY-based layout, presented on display 118. In anotherembodiment, different keyboard layouts may be implemented based on, forexample, the geographical region in which client computer system 110 isused (e.g., QWERTZ-based layout, AZERTY-based layout, etc.) presented ontouchscreen display 118. In general, GUI 114 may include graphicalicons, visual indicators, and other graphical elements by which the userof client computer system 110 interacts with application 116 to performvarious tasks. It should be understood that GUI 114 can dynamicallytransform (e.g., modulating size, changing orientation, etc.) based uponvarious parameters, such as a size and/or resolution of touchscreendisplay 118, an orientation of touchscreen display 118, viewing angle ofa user interacting with touchscreen display 118, etc. Analysis engine112 may generate a user interface (UI) structure that is arepresentation of GUI 114 prior to a modification operation, asdescribed in greater detail later in this specification.

Application 116 represents a program used by one or more users of clientcomputer system 110 to perform various tasks, via GUI 114. In oneembodiment, more than one application 116 can be executed on clientcomputer system 110. A first of the more than one application 116 mayimplement a unique GUI 114 configured specifically for the first of themore than one application 116. For example, application 116 may be avideo game executed on client computer system 110 that implements GUI114. In this instance, GUI 114 may include two buttons (e.g., anup-button and a down-button), as further described with regard to FIGS.4A-4C. In another example, application 116 may be a textual inputcomputer program, such that GUI 114 includes a virtual keyboard, asdescribed above. In general, application 116 may be used for variouspurposes by a user of client computer system 110, such as, generalproductivity, information retrieval, entertainment, communication, etc.

Touchscreen display 118 presents information provided by application 116to the user of client computer system 110. Touchscreen display 118 canbe an integrated touch-display of client computer system 110, forexample, the touch screen of a smartphone. In one embodiment,touchscreen display 118 is an external touch-display of client computersystem 110. Touchscreen display 118 can include sensing components tomonitor for touch events. Touch events performed on touchscreen display118 are associated with touch event information that can be received andanalyzed by analysis engine 112. A touch event on touchscreen display118 can be associated with an action performed by application 116 andcan represent an interaction with a portion of GUI 114, such as aninteractive graphical element of GUI 114. For example, touchscreendisplay 118 may implement various technologies to monitor for touchevents, such as resistive touch and capacitive touch technologies. Agranularity for a density map of touchscreen display 118 is based inpart, on the technology used to monitor for touch events. For example,if touchscreen display 118 implements a resistive touch technology, thena granularity for a density map is based on the resolution of theresistive touch technology (e.g., 4096×4096 DPI).

In one embodiment, touch event information may include, for example, atimestamp and/or duration for a touch event (i.e., an instance oftapping, swiping), an application associated with the touch event (i.e.,application 116), one or more panels of application 116 (i.e.,additional views of application 116), and one or more coordinates orother geometrical shape information that define an area of the touchevent.

Storage computer system 130 represents a platform for storing andmanaging information for each of the one or more users of clientcomputer system 110, their respective interactions with GUI 114, andother GUI and touchscreen interaction information. Storage computersystem 130 includes data store 132. As previously discussed, more thanone application 116 may be executed on client computer system 110. Inthis instance, more than one GUI 114 may correspond to each of the morethan one application 116. Accordingly, data store 132 stores touch eventinformation for each GUI 114 for the each of the more than oneapplication 116. In one embodiment, storage computer system 130 cantransmit information from data store 132 to client computer system 110and components therein via network 120. For example, application 116 mayrequest client computer system 110 for GUI 114 to be presented to a userof client computer system 110. In this instance, information related toGUI 114 may be exchanged between storage computer system 130 and clientcomputer system 110. The information may include modified dimensions orboundaries of one or more graphical elements of GUI 114, information toidentify a user of GUI 114, and sensing information. In anotherembodiment, data store 132 is a storage component implemented by clientcomputer system 110 (not depicted). In yet another embodiment, datastore 132 may include a UI structure (i.e., a previously modified GUI114), as described in greater detail later in this specification.

It should be understood that, for illustrative purposes, FIG. 1 does notshow other computer systems and elements which may be present whenimplementing embodiments of the present invention. For example, whileFIG. 1 shows a client computer system 110 and a single storage computersystem 130 and can also include additional client computer systems 110and storage computer systems 130 to support automated modification ofGUI 114 for multiple computer devices.

FIG. 2 is a flowchart 200 illustrating operational steps of analysisengine 112 for modifying GUI 114, in accordance with an embodiment ofthe present invention.

In step 202, analysis engine 112 determines that a monitoring intervalis complete. A monitoring interval is a time duration for analysisengine 112 to receive touch event information. In one embodiment,analysis engine 112 may indicate that a monitoring interval to receivetouch event information is complete if a number of touch events ontouchscreen display 118 performed by a user of client computer system110 exceeds a predefined threshold. In another embodiment, analysisengine 112 may indicate that the monitoring interval is complete if atime-out occurs. For example, a user of client computer system 110 maybe idle for a duration of time when interacting with GUI 114. In thisinstance, if the duration of idle time triggers a time-out, thenanalysis engine 112 may indicate that the monitoring interval iscomplete. It should be understood that, a time-out may not be triggeredif a number of recorded touch events are not sufficient to generate adensity map based on the associated touch event information.

In step 204, analysis engine 112 generates a density map using touchevent information received during the monitoring interval. In oneembodiment, analysis engine 112 aggregates touch event informationreceived by sensing components of touchscreen display 118, as previouslydiscussed. An exemplary method by which analysis engine 112 generates adensity map is described below in reference to FIGS. 3A, 3B, and 3C.

In step 206, analysis engine 112 identifies one or more portions of GUI114 for modification. In one embodiment, analysis engine compares a UIstructure with the density map. The phrase, “UI structure” as usedherein, refers to a representation of a previous or an original layoutof GUI 114. For example, a UI structure may be a representation of GUI114 prior to any modifications performed by analysis engine 112. Inanother example, a UI structure may be a representation of a mostrecently modified version of GUI 114. Accordingly, analysis engine 112may compare the density map with a UI structure to identify portions ofthe touchscreen display do not align with a threshold alignment value.

In step 208, analysis engine 112 modifies the portions of GUI 114 itidentified for modification. Modifications to GUI 114 may includemodifying a dimension of one or more graphical elements of GUI 114, suchas increasing a size of a graphical element that is most frequentlyinteracted with, by a user of client computer system 110. Anothermodification to GUI 114 may involve decreasing a size of a graphicalelement that is not frequently interacted with, by a user of clientcomputer system 110. Other modifications to GUI 114 may involvetransposing one or more portions of a generated density map. Forexample, a user of client computer system 110 may frequently tap asquare graphical element, but a density map generated from touch eventinformation may indicate that a circular portion encompassing an area ofthe square graphical element is frequently interacted with by the userof client computer system. In this instance, the square graphicalelement may be modified into the circular area, as indicated by thedensity map. Accordingly, modifications to GUI 114 may improveaccessibility for the user of client computer system 110 to one or moregraphical elements of GUI 114, and may improve ease of use for themodified GUI 114 used by the user of client computer system 110interacting with application 116. In one embodiment, analysis engine 112can present the modified GUI 114 on touchscreen display 118 of clientcomputer system 110. In another embodiment, a new monitoring intervalmay restart. In this instance, the received touch event information forthe new monitoring interval may result in an additional modifications ofGUI 114. In yet another embodiment, a notification may be prompted to auser of client computer system 110, indicating that GUI 114 has beenmodified for the user of client computer system 110. In this instance,the user of client computer system 110 may accept, reject, or modify themodified GUI 114. In certain embodiments, modification of a portion ofGUI 114 may include changing the dimensions of a graphical element ofGUI 114, and modifying the corresponding touch screen area to differentdimensions and/or a different screen position relative to the graphicalelement.

FIGS. 3A-3C are diagrams representing graphical elements of GUI 114. Inthis embodiment, FIGS. 3A-3C present two buttons. For example, FIG. 3Aillustrates up-button 302 and down-button 304. In one embodiment, a userof client computer system 110 may perform a touch event on touchscreendisplay 118 to interact with buttons 302 and 304 of GUI 114. Each areaof up-button 302 and down-button 304 displayed on touchscreen display118 includes a plurality of dots, wherein each dot is an area configuredto receive a touch event performed on touchscreen display 118. It shouldbe understood that, FIGS. 3A-3C are provided for illustrative purposes,and that FIGS. 3A-3C may include a smaller or larger number of graphicalelements. A shape of each graphical element (e.g., up-button 302,down-button 304, etc.) can be a regular and/or an irregular shape.

FIG. 3A is a block diagram 300 representing graphical elements of GUI114 for application 116. For example, block diagram 300 may representtwo graphical elements of GUI 114 used to interact with application 116of client computer system 110 (e.g., a video game). In this instance,up-button 302 may represent a virtual up-key of GUI 114, and down-button304 may represent a virtual down-key of GUI 114. A user of clientcomputer system 110 may perform a touch event on touchscreen display 118to interact with up-button 302, such that application 116 performs ajump command in the video game. Similarly, a user of client computersystem 110 may perform a touch event on touchscreen display 118 tointeract with down-button 304, such that application 116 performs a duckcommand for the video game. In another example, application 116 may bean image viewing application, wherein a user of client computer system110 may interact with up-button 302 to scroll upwards in application 116and the user of client computer system 110 may interact with down-button304 to scroll downwards in application 116.

FIG. 3B is a block diagram 320 representing a density map for the twographical elements of GUI 114. As previously discussed, a density map isgenerated from touch event information received during a monitoringinterval. FIG. 3B identifies one or more graphical elements of GUI 114presented in FIG. 3A most frequently interacted with, by a user ofclient computer system 110 when using application 116. In thisembodiment, section 306, section 308, and section 310 represent threeportions of GUI 114, wherein each of the three portions vary infrequency with respect to a number of instances of touch events receivedduring a monitoring interval. The density map shows section 306 as thelargest of sections 306, 308, and 310, which may indicate that the userof client computer system 110 interacted with an area of section 306least frequently compared to sections 308 and 310. Section 308 isillustrated as the second largest of sections 306, 308, and 310, whichmay indicate that the user of client computer system 110 interacted withsection 308 second most frequently compared to sections 306 and 310.Section 310 is illustrated as the smallest of sections 306, 308, and310, which may indicate that the user of client computer system 110interacted with section 310 most frequently compared to sections 306 and308. In this embodiment, up-button 302 and down-button 304 are disposedin GUI 114 directly next to each other in FIG. 3A. Accordingly, a userof client computer system 110 can interact with buttons 302 and 304, andtouch event information associated with the interactions contribute totouch event information used to generate FIG. 3B. In this instance,additional information may be used to determine a contribution amountfrom a particular graphical element of GUI 114. Stated differently,analysis engine 112 may use additional information to ensure that touchevent information that is used to modify graphical elements of GUI 114are used appropriately. This information is used by analysis engine 112,such that boundaries of neighboring graphical elements, such asup-button 302 and down-button 304, are only being modified in accordancewith touch event information particular to the graphical element beingmodified.

FIG. 3C is a block diagram 340 representing graphical elements of amodified GUI 114. In one embodiment, analysis engine 112 may comparegraphical elements of a UI structure, such as a representation of FIG.3A, with a density map, such as FIG. 3B, to generate graphical elementsthat are a part of a modified GUI 114. In this embodiment, analysisengine 112 identified section 306, section 308, and section 310 in thedensity map as portions of GUI 114 most frequently interacted with whena user of client computer system 110 attempts to interact with up-button302 of GUI 114. In this instance, analysis engine 112 determines thatup-button 302 of FIG. 3A may require a larger area in accordance withthe density map presented in FIG. 3B. Accordingly, up-button 302 fromFIG. 3A is modified by increasing an associated area, such that thegraphical element represented by up-button 302 is now up-button 312 anddown-button 304 from FIG. 3A is now down-button 314. In anotherembodiment, an area configured to receive touch events for a particulargraphical element of GUI 114 can be modified. For example, an area ofthe graphical element, such as an area of up-button 302, may not bemodified into up-button 312. Instead, an area that contains sections306, 308, and 310 may be configured to perform an action associated withup-button 302. In this example, a modified GUI 114 may involve receivinga touch event performed on an area of section 306 and modifying an areaconfigured to receive touch events, such that the touch event on an areaof section 306 is associated with an interaction on up-button 302.

FIG. 4 is a block diagram of internal and external components of acomputer system 400, which is representative the computer systems ofFIG. 1, in accordance with an embodiment of the present invention. Itshould be appreciated that FIG. 4 provides only an illustration of oneimplementation and does not imply any limitations with regard to theenvironments in which different embodiments may be implemented. Ingeneral, the components illustrated in FIG. 4 are representative of anyelectronic device capable of executing machine-readable programinstructions. Examples of computer systems, environments, and/orconfigurations that may be represented by the components illustrated inFIG. 4 include, but are not limited to, personal computer systems,server computer systems, thin clients, thick clients, laptop computersystems, tablet computer systems, cellular telephones (e.g., smartphones), multiprocessor systems, microprocessor-based systems, networkPCs, minicomputer systems, mainframe computer systems, and distributedcloud computing environments that include any of the above systems ordevices.

Computer system 400 includes communications fabric 402, which providesfor communications between one or more processors 404, memory 406,persistent storage 408, communications unit 412, and one or moreinput/output (I/O) interfaces 414. Communications fabric 402 can beimplemented with any architecture designed for passing data and/orcontrol information between processors (such as microprocessors,communications and network processors, etc.), system memory, peripheraldevices, and any other hardware components within a system. For example,communications fabric 402 can be implemented with one or more buses.

Memory 406 and persistent storage 408 are computer-readable storagemedia. In this embodiment, memory 406 includes random access memory(RAM) 416 and cache memory 418. In general, memory 406 can include anysuitable volatile or non-volatile computer-readable storage media.Software is stored in persistent storage 408 for execution and/or accessby one or more of the respective processors 404 via one or more memoriesof memory 406.

Persistent storage 408 may include, for example, a plurality of magnetichard disk drives. Alternatively, or in addition to magnetic hard diskdrives, persistent storage 408 can include one or more solid state harddrives, semiconductor storage devices, read-only memories (ROM),erasable programmable read-only memories (EPROM), flash memories, or anyother computer-readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 408 can also be removable. Forexample, a removable hard drive can be used for persistent storage 408.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer-readable storage medium that is also part of persistent storage408.

Communications unit 412 provides for communications with other computersystems or devices via a network (e.g., network 120). In this exemplaryembodiment, communications unit 412 includes network adapters orinterfaces such as a TCP/IP adapter cards, wireless Wi-Fi interfacecards, or 3G or 4G wireless interface cards or other wired or wirelesscommunication links. The network can comprise, for example, copperwires, optical fibers, wireless transmission, routers, firewalls,switches, gateway computers and/or edge servers. Software and data usedto practice embodiments of the present invention can be downloaded toclient computer system 110 through communications unit 412 (e.g., viathe Internet, a local area network or other wide area network). Fromcommunications unit 412, the software and data can be loaded ontopersistent storage 408.

One or more I/O interfaces 414 allow for input and output of data withother devices that may be connected to computer system 400. For example,I/O interface 414 can provide a connection to one or more externaldevices 420 such as a keyboard, computer mouse, touch screen, virtualkeyboard, touch pad, pointing device, or other human interface devices.External devices 420 can also include portable computer-readable storagemedia such as, for example, thumb drives, portable optical or magneticdisks, and memory cards. I/O interface 414 also connects to display 422.

Display 422 provides a mechanism to display data to a user and can be,for example, a computer monitor. Display 422 can also be an incorporateddisplay and may function as a touch screen, such as a built-in displayof a tablet computer.

Referring now to FIG. 5, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 6 are intended to be illustrative only and that cloud computingnodes 10 and cloud computing environment 50 can communicate with anytype of computerized device over any type of network and/or networkaddressable connection (e.g., using a web browser).

Referring now to FIG. 6, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 5) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 6 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and GUI modification environment 96.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in FIGS. 1-6 illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in FIGS. 1-6. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in FIGS. 1-6 illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in FIGS. 1-6. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The terminology used herein was chosen to best explain the principles ofthe embodiment, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds). A cloud computing environment is serviceoriented with a focus on statelessness, low coupling, modularity, andsemantic interoperability. At the heart of cloud computing is aninfrastructure comprising a network of interconnected nodes.

What is claimed is:
 1. A method for automatically modifying a graphicaluser interface (GUI), the method comprising: receiving, by one or morecomputer processors, information that describes one or more touch eventsperformed on a touchscreen display to interact with the GUI of asoftware application; generating, by one or more computer processors, atouch event density map based on a frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display for each of the one or more graphical elements ofthe GUI, wherein each of the one or more graphical elements of the GUIhaving a shape of a physical area that identifies an area on thetouchscreen display associated with the each of the one or moregraphical elements and a visually displayed shape having a four squareshaped graphical element that is displayed on the touchscreen display;identifying, by one or more computer processors, one or more portions ofthe touchscreen display that received a highest density of touch eventsbased on analyzing the one or more portions of the touch event densitymap to identify one or more portions of the touchscreen display thatreceives the highest density of touch events; identifying, by one ormore computer processors, one or more graphical elements of the GUIassociated with the one or more identified portions of the touchscreendisplay; and responsive to determining that the one or more graphicalelements of the GUI associated with the one or more identified portionsof the touchscreen display do not align with a threshold alignment valuemodifying, by one or more computer processors, the shape of the physicalarea of the one or more graphical elements of the GUI associated withthe one or more identified portions of the touchscreen display, whereinthe visually displayed shape of the one or more graphical elements ofthe GUI are not modified, a size of each of the one or more graphicalelements of the GUI based on a frequency of a user interaction with eachof the one or more graphical elements of the GUI, and a position of thephysical area of the one or more graphical elements relative to the oneor more graphical elements of the touchscreen display area, wherein thevisually displayed shape of the one or more graphical elements of theGUI are not modified.
 2. The method of claim 1, wherein generating atouch event density map based on the frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display comprises: receiving, by one or more computerprocessors, a touch event on the touchscreen display; and responsive todetermining that a monitoring interval to record a touch event on thetouchscreen display is complete, aggregating, by one or more computerprocessors, each touch event of the monitoring interval to generate thetouch event density map based on the frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display.
 3. The method of claim 2, wherein a monitoringinterval to record touch event on the touchscreen display is completebased on at least one of: a number of touch events received on thetouchscreen display exceeds a predefined threshold, and a time-outoccurring.
 4. The method of claim 1, wherein the shape of the physicalarea having a circular area based on the touch event density map,wherein the circular area encompasses the area of the GUI associatedwith the one or more identified portions of the touchscreen display. 5.The method of claim 1, wherein a granularity for a touch event densitymap is dependent on a number of areas implemented by the touchscreendisplay that are configured to receive information that describes one ormore touch events.
 6. The method of claim 1, wherein responsive todetermining that the one or more graphical elements of the GUIassociated with the one or more identified portions of the touchscreendisplay do not align with the threshold alignment value comprises:analyzing, by one or more computer processors, the touch event densitymap to determine that the one or more graphical elements of the GUIassociated with the one or more portions of the touchscreen display thatdo not align with the threshold alignment value.
 7. The method of claim6, further comprising: comparing, by one or more computer processors,the GUI with the one or more identified portions of the touchscreendisplay to identify the one or more graphical elements of the GUIassociated with the one or more identified portions of the touchscreendisplay that do not align with the threshold alignment value.
 8. Acomputer program product for automatically modifying a graphical userinterface (GUI), the computer program product comprising: one or morecomputer readable storage media and program instructions stored on theone or more computer readable storage media, the program instructionscomprising: program instructions to receive information that describesone or more touch events performed on a touchscreen display to interactwith the GUI of a software application; program instructions to generatea touch event density map based on a frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display for each of the one or more graphical elements ofthe GUI, wherein each of the one or more graphical elements of the GUIhaving a shape of a physical area that identifies an area on thetouchscreen display associated with the each of the one or moregraphical elements and a visually displayed shape having a four squareshaped graphical element that is displayed on the touchscreen display;program instructions to identify one or more portions of the touchscreendisplay that received a highest density of touch events based onanalyzing the one or more portions of the touch event density map toidentify one or more portions of the touchscreen display that receivesthe highest density of touch events; program instructions to identifyone or more graphical elements of the GUI associated with the one ormore identified portions of the touchscreen display; and programinstructions to, responsive to determining that the one or moregraphical elements of the GUI associated with the one or more identifiedportions of the touchscreen display do not align with a thresholdalignment value, modify the shape of the physical area of the one ormore graphical elements of the GUI associated with the one or moreidentified portions of the touchscreen display, wherein the visuallydisplayed shape of the one or more graphical elements of the GUI are notmodified, a size of each of the one or more graphical elements of theGUI based on a frequency of a user interaction with each of the one ormore graphical elements of the GUI, and a position of the physical areaof the one or more graphical elements relative to the one or moregraphical elements of the touchscreen display area, wherein the visuallydisplayed shape of the one or more graphical elements of the GUI are notmodified.
 9. The computer program product of claim 8, wherein programinstructions to generate a touch event density map based on thefrequency of touch events on the touchscreen display as a function oftouch event position on the touchscreen display comprise: programinstructions to receive a touch event on the touchscreen display; andprogram instructions to, responsive to determining that a monitoringinterval to record a touch event on the touchscreen display is complete,aggregate each touch event of the monitoring interval to generate thetouch event density map based on the frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display.
 10. The computer program product of claim 9,wherein a monitoring interval to record touch event on the touchscreendisplay is complete based on at least one of: a number of touch eventsreceived on the touchscreen display exceeds a predefined threshold, anda time-out occurring.
 11. The computer program product of claim 8,wherein the shape of the physical area having a circular area based onthe touch event density map, wherein the circular area encompasses thearea of the GUI associated with the one or more identified portions ofthe touchscreen display.
 12. The computer program product of claim 8,wherein a granularity for a touch event density map is dependent on anumber of areas implemented by the touchscreen display that areconfigured to receive information that describes one or more touchevents.
 13. The computer program product of claim 8, wherein programinstructions to, responsive to determining that the one or moregraphical elements of the GUI associated with the one or more identifiedportions of the touchscreen display do not align with a thresholdalignment value comprise: program instructions to analyze a touch eventdensity map to determine that the one or more graphical elements of theGUI associated with the one or more portions of the touchscreen displaythat do not align with the threshold alignment value.
 14. The computerprogram product of claim 13, wherein the program instructions stored onthe one or more computer readable storage media further comprise:comparing, by one or more computer processors, the GUI with the one ormore identified portions of the touchscreen display to identify the oneor more graphical elements of the GUI associated with the one or moreidentified portions of the touchscreen display that do not align withthe threshold alignment value.
 15. A computer system for automaticallymodifying a graphical user interface (GUI), the computer systemcomprising: one or more computer processors; one or more computerreadable storage media; program instructions stored on the one or morecomputer readable storage media for execution by at least one of the oneor more processors, the program instructions comprising: programinstructions to receive information that describes one or more touchevents performed on a touchscreen display to interact with the GUI of asoftware application; program instructions to generate a touch eventdensity map based on a frequency of touch events on the touchscreendisplay as a function of touch event position on the touchscreen displayfor each of the one or more graphical elements of the GUI, wherein eachof the one or more graphical elements of the GUI having a shape of aphysical area that identifies an area on the touchscreen displayassociated with the each of the one or more graphical elements and avisually displayed shape having a four square shaped graphical elementthat is displayed on the touchscreen display; program instructions toidentify one or more portions of the touchscreen display that received ahighest density of touch events based on analyzing the one or moreportions of the touch event density map to identify one or more portionsof the touchscreen display that receives the highest density of touchevents; program instructions to identify one or more graphical elementsof the GUI associated with the one or more identified portions of thetouchscreen display; and program instructions to, responsive todetermining that the one or more graphical elements of the GUIassociated with the one or more identified portions of the touchscreendisplay do not align with a threshold alignment value, modify the shapeof the physical area of the one or more graphical elements of the GUIassociated with the one or more identified portions of the touchscreendisplay, wherein the visually displayed shape of the one or moregraphical elements of the GUI are not modified, a size of each of theone or more graphical elements of the GUI based on a frequency of a userinteraction with each of the one or more graphical elements of the GUI,and a position of the physical area of the one or more graphicalelements relative to the one or more graphical elements of thetouchscreen display area, wherein the visually displayed shape of theone or more graphical elements of the GUI are not modified.
 16. Thecomputer system of claim 15, wherein program instructions to generate atouch event density map based on the frequency of touch events on thetouchscreen display as a function of touch event position on thetouchscreen display comprise: program instructions to receive a touchevent on the touchscreen display; and program instructions to,responsive to determining that a monitoring interval to record a touchevent on the touchscreen display is complete, aggregate each touch eventof the monitoring interval to generate the touch event density map basedon the frequency of touch events on the touchscreen display as afunction of touch event position on the touchscreen display.
 17. Thecomputer system of claim 16, wherein a monitoring interval to recordtouch event on the touchscreen display is complete based on at least oneof: a number of touch events received on the touchscreen display exceedsa predefined threshold, and a time-out occurring.
 18. The computersystem of claim 15, wherein the shape of the physical area having acircular area based on the touch event density map, wherein the circulararea encompasses the area of the GUI associated with the one or moreidentified portions of the touchscreen display.
 19. The computer systemof claim 15, wherein a granularity for a touch event density map isdependent on a number of areas implemented by the touchscreen displaythat are configured to receive information that describes one or moretouch events.
 20. The computer system of claim 15, wherein programinstructions to, responsive to determining that the one or moregraphical elements of the GUI associated with the one or more identifiedportions of the touchscreen display do not align with a thresholdalignment value comprise: program instructions to analyze a touch eventdensity map to determine that the one or more graphical elements of theGUI associated with the one or more portions of the touchscreen displaythat do not align with the threshold alignment value.